Pupilometer

ABSTRACT

A pupilometer comprises image capturing means, illumination means comprising two spaced apart light sources, stimulation means, and image processing software, the illumination means generating and emitting light of a first wave-length, and the stimulation means generating and emitting light of a second wavelength. The illumination means is arranged to one or both sides of said image capturing means and, in use, shines light towards the eyeball, the image processing software receiving data from the image capturing means, and by processing said data according to an algorithm establishes the distance between the surface of the eyeball and the camera.

FIELD OF THE INVENTION

This invention relates to an apparatus commonly known as a pupilometer.

BACKGROUND OF THE INVENTION

In the neurological assessment of an unconscious patient, pupil responseis known to be a vital aspect of the diagnostic process. Regularassessment of the size, reactivity to light and equality of pupils isessential for early recognition of neurological deterioration insituations where intra-cranial pathology is a threat. As such thisassessment is regularly carried out in paramedic, intensive and highdependency care situations.

The current method of practice is to manually measure these aspectsusing a bright light, which stimulates reactivity of the pupil and makea note of the dilation compared to the original size of the pupil Actualmeasurements taken are then compared with a card having different pupilsizes mated thereon. This method of assessment is time consuming, andsubjective.

Pupilometers have been developed for use in the assessment of eye shapeand condition, monitoring tiredness, and in the detection of drugs oralcohol in a person.

A hand-held pupilometer is described in U.S. Pat. No. 6,022,109 DalSante). This pupilometer detects and measures pupil diameter and pupilresponse to a light stimulus. Also described is software to permit thediagnosis of alcohol or drug presence. However, use of this pupilometerrequires the active participation of the user.

Another hand-held pupilometer is described in U.S. Pat. No. 6,199,985(Anderson). This patent describes a method for measuring optical poweroutput from the pupil. However, the pupilometer described in the patentrequires complex optometric components.

Another hand-held pupilometer is described in U.S. Pat. No. 6,260,968(Stark). The device described includes an LCD display via which promptsto the operator are given. The pupilometer described in this patent usesa “flying spot” algorithm to establish a circumference fitting thepupil, and the pupil radius. The pupilometer includes software to aiddiagnosis. Again, the pupilometer described in this patent requirescomplex optometric components.

It would therefore be desirable to provide an improved pupilometer.

SUMMARY OF THE INVENTION

According to one aspect of the invention there is provided apupilometer.

According to another aspect of the invention, there is provided imageprocessing software.

The software may be embodied on a record medium, stored in a computermemory, embodied in read only memory, or carried on an electricalsignal.

According to another aspect of the invention, there is provided aprocess for obtaining pupil image information.

According to another aspect of the invention, there is provided ahand-held pupilometer.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which illustrate exemplary embodiments of pupilometersaccording to the invention:

FIG. 1 is a schematic representation of a perfect eye;

FIG. 2 is a block diagram of a pupilometer;

FIG. 3 is a schematic representation of an eyeball;

FIG. 4 is an image of an eye under ambient fight;

FIG. 5 is an image of an eye under infra-red light;

FIG. 6 shows a raw image taken by the pupilometer before any imageprocessing has taken place;

FIG. 7 shows the image of FIG. 6 after the identification of darkpixels;

FIG. 8 is a table used to identify an edge;

FIG. 9 shows the image of FIG. 7 after identification of the pupil edge;

FIG. 10 shows the image of FIG. 9 at the beginning of a spiral search;

FIG. 11 shows the image of FIG. 9 with adjoining pupil edge pixelsconnected to one another;

FIG. 12 is a table illustrating a recursive flood-fill algorithm;

FIG. 13 shows the image of FIG. 11 with the rectangular dimension of thepupil identified;

FIG. 14 shows an image of a part of an eye close to the pupil whensubjected to highlights from infra-red LED's of the pupilometer;

FIG. 15 shows the image of FIG. 14 with possible highlights marked;

FIG. 16 shows the comparison of highlight vertical co-ordinates in FIG.15;

FIG. 17 shows identification of the highlights of FIG. 16 with theclosest vertical alignment;

FIG. 18 shows the image of FIG. 17 with the distance between the twovalid highlights marked;

FIG. 19 is a graph showing the reaction over time of a pupil diameter toa light stimulation;

FIG. 20 is a schematic cross-section of a hand-held pupilometer;

FIG. 21 is a plan view of the pupilometer illustrated in FIG. 20;

FIG. 22 is a block diagram of the pupilometer shown in FIGS. 20 and 21;

FIG. 23 is a schematic representation of a pupilometer of the inventionin close proximity to an eye; and

FIG. 24 shows an image of a part of an eye close to the pupil whensubjected to highlights from infra-red LED's of the pupilometer;

FIG. 25 shows the image of FIG. 14 with the highlights marked;

FIG. 26 is a schematic representation of the marked highlights of FIG.15;

FIG. 27 shows the image of FIG. 14 with the distance from the highlightto the centre marked; and

FIG. 28 shows the image of FIG. 14 with the distance between the twohighlights marked.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 2 illustrates the components of an embodiment of the pupilometer.The pupilometer comprises a camera board 10 including a camera, which inthe example is a CMOS (Complementary Metal Oxide Semiconductor) camera11, a filter 12, which in the example is an infra-red pass filter, apair of infra-red light emitting diodes (IR LED's) 13, and a lightemitting diode (LED) 14 for emitting white light The camera 11 and LED's13, 14 are mounted on a board, which in the example is a printed circuitboard 15, the filter 12 being mounted in front of the lens of the camera11.

The camera board 10 is connected by suitable cabling to a control board20, which mounts an analogue interface 21, a micro-controller 22, amemory 23 and a Universal Serial Bus (USB) interface 24. The analogueinterface 21, memory 23 and USB interface 24 are each connected to themicro-controller 22 by suitable cabling 25. The analogue interface 21receives an analogue video signal from the camera board 10 and convertssaid signal into a digital form The micro-controller 22 provides controlsignals for image acquisition from the camera board 10, and transmissionof image data to a computer programmed with custom pupil detection andmeasurement software, which in the example is a laptop computer 26connected to the micro controller 22 via a USB interface 24. However,the computer programmed with custom pupil detection and measurementsoftware could easily form part of a hand held pupilometer device. Sucha device is described with reference to FIGS. 20 to 22.

The control board 20 also mounts a memory module 23 which providesadditional static RAM for storage of image data acquired from the cameraboard 10 prior to transmission of the image data to the computer 26,with the USB interface 24 providing a physical interface for theconversion and transmission of image frames to the computer 26 over astandard USB interface.

The computer 26 of the example runs the operating system, “MicrosoftWindows 95”, and custom software which detects and measures the pupil inthe images generated by the camera.

Referring now to FIG. 3, the IR LED's 13 shine light towards the eyeball30, but to the sides of the pupil 31. By virtue of illuminating theeyeball by shining light to the sides of the pupil 31, most of the raysof light entering the pupil are internally reflected and absorbed by theretina, and thus the camera only sees light reflected from the surfaceof the eye, with the pupil appearing as a dark area.

The purpose of the infra-red pass filter 12 is to stop all visible lightentering the camera 11, which eliminates the effects of ambient lightconditions, thereby permitting accurate control of the instrument.

FIGS. 4 and 5 illustrate the difference in appearance of an eye underambient light conditions (see FIG. 4) and infra red lights (see FIG. 5).In FIG. 5, the contrast between the pupil 3 and the iris 2 is increasedcompared to FIG. 4. Also, there is much less surrounding detail in FIG.5 compared to FIG. 4.

The reflections from the IR LED's 13 can be seen clearly in FIG. 5, andthe distance between these specular highlights is used as measure of thedistance from the camera to the eye (the closer the IR LED's are to theeye, the further apart are the highlights).

Referring now to FIG. 23, the eye 1 is illuminated by light in theinfrared spectrum of light beams 32 emitted by the infrared LED's 13.The camera 11 sees highlights 33 on the surface of eyeball 30. For thepupilometer to generate an output of pupil size, separate highlightsfrom each infra-red LED 13 must be detectable, and therefore must bewithin a certain distance of the surface of the eyeball. The extremes ofthe light beams 32 are illustrated by dotted lines. Clearly, if theinfra-red LED's are too far away from the surface of the eyeball, theywill overlap, in which case two spaced apart highlights 33 would not beappear on the surface of the eyeball. Conversely, if the infra-red LED'sare too close to the surface of the eyeball, then the highlights will beso far apart as to be located in close proximity to the eyelids, ratherthan in the central region of the surface of the eyeball 30. In such asituation, the two highlights cannot be detected. Therefore, if thepupilometer is outside the range r_(min)-r_(max), where r_(min) is theminimum distance from the infra-red LED's to the surface of the eyeball30, and r_(max) is the maximum distance of the infra-red LED's to thesurface of the eyeball 30, two separate highlights cannot be detected,and the pupilometer software produces a range error signal. Thealgorithm restarts the ranging step after pupil detection in the nextcaptured image.

Referring now to FIGS. 20 to 22, there is shown a hand-held pupilometer,which comprises a camera board 110 including a camera, which in theexample is a CMOS (Complementary Metal Oxide Semiconductor) camera 111,a filter 112, which in the example is an infra-red pass filter, a pairof infra-red light emitting diodes (IR LED's) 113, and a light emittingdiode (LED) 114 for emitting white light The camera 111 and LED's 113,114 are mounted on a board, which in the example is a printed circuitboard 115, the filter 112 being mounted in front of the lens of thecamera 111.

The camera board 110 is connected to a control board 120, which mountsan analogue interface 121, a micro-controller 122, a memory 123 and acomputer interface 106. The analogue interface 121, memory 123 andcomputer interface 106 are each connected to the micro-controller 122 bysuitable cabling 125. The analogue interface 121 receives an analoguevideo signal from the camera board 110 and converts said signal into adigital form The micro-controller 122 provides control signals for imageacquisition from the camera board 110. Further, the microcontroller 122transmits image data to, and runs, custom pupil detection andmeasurement software.

As mentioned above, the control board 120 also mounts a memory module123 which provides additional static RAM for storage of image dataacquired from the camera board 110 for use by the custom pupil detectionand measurement software of the micro-controller.

The computer interface 106 provides a physical interface fortransmission of data to an external computer. It may be desirable tostore test results in patients' notes, or for research purposes, andwhilst the hand-held device 100 has sufficient memory to record a numberof results, to use the device continually, the memory 123 must becleared from time to time.

As with the device described with reference to FIG. 3, the IR LED's 113shine light towards the eyeball 30, but to the sides of the pupil 31. Byvirtue of illuminating the eyeball by shining light to the sides of thepupil 31, most of the rays of light entering the pupil are internallyreflected and absorbed by the retina, and thus the camera only seeslight reflected from the surface of the eye, with the pupil appearing asa dark area.

Pupilometer Software

The main function of the software is to interpret the image of the eyeand detect, or classify, the pupil within that image. The software wasdeveloped using Borland Delphi and in the example executes under theMicrosoft Windows operating system.

The basic requirement is the ability to detect a circle (i.e. the pupil)within the image and known algorithms available for the performance ofthis task include the Hough transform, parametric matching and neuralnetwork classification. However, these methods are computationallyintensive and require a floating-point numeric processor in order toachieve optimal performance.

One aim of the invention to provide a standalone hand-held pupilometer.This means that a relatively low specification microprocessor must beused and therefore the algorithm of the invention is a simplemulti-stage classification algorithm, which uses integer mathematicalfunctions to classify the pupil within the image.

Referring now to FIG. 1, there is shown a model of a perfect eye, i.e.the iris 2 is at the centre of the eyeball 1, with the pupil 3 being atthe centre of the iris 2. Further, both the pupil 3 and the iris 2 areperfect circles, the boundary 4 between the iris 2 and the pupil 3 issharp, and the darkest region of the eye is the pupil 3.

The software of the invention makes certain assumptions based on themodel of the perfect eye described above, those assumptions being:

1) The pupil will be the darkest area of the image;

2) The pupil—iris boundary will have the sharpest edge;

3) The pupil—iris boundary will be elliptical.

The software provides three principal functions;

-   -   1. Pupil classification: the detection and measurement of the        pupil within the image of the eye.    -   2. Ranging: the detection and measurement of the IR LED        reflections on the eye surface allowing calculation of distance        from camera to eye.    -   3. Stimulation: measurement of the pupil reflex action to light        stimulation.

Pupil Classification

The classification algorithm of the invention provides for thedifferentiation of the pupil from other dark areas of the image, such asshadows, and from interference within the pupil boundary, for exampleeyelashes and highlights.

The control board 20 transmits a new image every 200 ms via the USBinterface 24. The image is returned as a two-dimensional (128×128 pixel)array of 6 bit values, with each value representing the greyscaleintensity of the relevant image pixel in the range 0 to 63. This imageis then subjected to the following processing steps:

-   -   1) As the raw image array is read into the Delphi program, the        values of the darkest (Vdark) and the lightest (Vlight) pixels        are calculated and stored. Threshold levels are then calculated        using these values; Tdark=Vdark+4 and Tlight=Vlight−2—see FIG.        6.    -   2) All image pixels with values of less than or equal to this        dark threshold (Td) are assigned to the PUPIL class—see FIG. 7.    -   3) The edge values across each of these PUPIL class pixels are        calculated using the simple gradient algorithm        |P4−P0|+|P4−P1|+|P4−P2|+|P4−P5|+|P4−P8|+|P4−P7|+|P4−P6|+|P4−P3|=G        the gross radial gradient. This algorithm produces the gross        radial gradient (G) across the central pixel (P4)—see FIG. 8.    -   4) All image pixels with edge values (G) of greater than or        equal to 8 are assigned to the PUPIL_EDGE class—see FIG. 9. The        pupil edge value of 8 was selected using empirical methods as a        value discriminating valid edge pixels.    -   5) In order to locate an area of PUPIL_EDGE pixels large enough        to be the actual pupil, a spiral search is initiated from the        centre of the image (or the centre of a valid pupil from the        previous frame to improve the speed of location), is used to        locate the first PUPIL_EDGE pixel and this is assumed to lie on        the pupil boundary—see FIG. 10.    -   6) When the search locates a PUPIL_EDGE pixel—see FIG. 10, All        adjoining PUPIL_EDGE pixels are connected using a recursive        flood fill algorithm The fill algorithm also tracks the numbers        and extents of the adjoining pixels, from which the width and        height of pupil region are derived—see FIG. 11. If the fill        connects more than 16 pixels, the area is designated as being        the pupil boundary area and the algorithm continues to step 7.        If the fill connects less than 16 pixels, the area is designated        as being too small to be the pupil and the spiral search (5)        continues outwards until another PUPIL EDGE region is found or        the extents of the image are reached. If the pupil boundary area        has not been located by the end of the spiral search, the        algorithm restarts at step 1 with the next captured image.

In steps 5 and 6, every time the spiral search hits a PUPIL_EDGE theregion is flood filled to try to find a region large enough to be thepupil. When the pupil is identified, the spiral search exits.

Recursive Flood Fill

The fill algorithm sets the target pixel and tests each of its fourneighbours, in north-west-south-east order, for another PUPIL EDGEpixel. As soon as such a pixel is found, the algorithm re-calls itselfwith this new pixel as its target. An enlarged view of a typical fillpattern is shown in FIG. 12. The first branch is filled by the routinecalling itself nine ties and stops when no further PUPIL EDGE pixels arefound, the second branch (dotted arrows) search then starts. In thisway, the routine continues until all adjoining PUPIL EDGE pixels havebeen set—see FIG. 12.

The rectangular dimension of the pupil boundary area is calculated fromthe extents of the flood fill and an ellipse consisting of thirty-twopoints is fitted inside this rectangle. If twelve or more of thesepoints hit a PUPIL EDGE pixel the region is classified as the PUPIL andthe range detection phase begins; if not the search re-starts with thenext captured image. The pupil diameter is defined as the maximumdiameter of the ellipse—see FIG. 13.

Ranging

When a valid pupil has been classified it is known that the highlightsfrom the infra red LED's will appear in the image within close proximityto the pupil. Therefore to improve speed of calculation and removal ofartefacts from eyelids etc, only the area around the pupil is searched.

A first procedure for ranging is illustrated in FIGS. 14 to 18, and isdescribed below. The search identifies discrete groups of pixels whichcould belong to valid highlights

With reference to FIG. 14, a search area 84 wide by 64 pixels highcentred around the pupil is scanned to identify pixels with values ofgreater than or equal to the previously assigned threshold Tlight, theseare classed as HIGHLIGHT_TEST pixels. When such a pixel is found, aflood fill of adjoining HIGHLIGHT_TEST pixels is initiated during whichthe number of pixels and centre co-ordinates of the fill area isrecorded.

Possible highlights are defined as fill areas with pixel counts withinthe range 4 to 256 pixels, and FIG. 15 illustrates the identification ofsuch areas. These areas are designated as possible valid highlights andtheir centre co-ordinates and pixel counts are stored in an array. Inorder to minimise memory usage, a maximum of 16 areas are allowed.

As shown in FIG. 16, when the whole search area has been scanned and twoor more possible highlight areas identified, the vertical positions ofall areas are compared in order to identify the two areas with theclosest vertical alignment.

FIG. 17 shows the identification of two such areas.

If less than two or no suitably aligned highlights have been identified,the algorithm is unable to derive range information, and a “range error”signal is generated and the pupil detection phase restarts on the nextcaptured image.

FIG. 18 illustrates the final range step, where with both validhighlight areas identified, the horizontal distance between theircentres and the geometry of the infra-red LED position and the lightemitted thereby allow the software to calculate accurately the distanceof the pupilometer from the surface of the patients eyeball. If thedistance to the eyeball is outside the valid detection range r_(min) tor_(max), the algorithm will generate a “range error” signal and thepupil detection phase restarts on the next captured image.

A second procedure for ranging is illustrated in FIGS. 24 to 28 isdescribed below.

With reference to FIG. 24, an area twelve pixels above and below thepupil is scanned to find the BRIGHTEST pixel level.

With reference to FIG. 25, the area is rescanned and pixels with a valuegreater than BRIGHTEST-8 are marked as HIGHLIGHT pixels. The maximum x/yextent of these HIGHLIGHT pixels is recorded and the centre of theextents is calculated.Centre X=(Max Highlight X−Min Highlight X)/2Centre Y=(Max Highlight Y−Min Highlight Y)/2

FIG. 26 illustrates horizontal lines of pixels, starting from the centrepixel (PASS 1) and expanding one pixel vertically above and below thecentre line (PASS 2 . . . ), which are scanned to the right hand extentsuntil a HIGHLIGHT pixel is found.

FIG. 27 shows the HIGHLIGHT area flood-filled, with the centre of thearea calculated from the extents of the flood-fill. Steps 16 and 17 arethen repeated for the pixels on the left-hand side of the centre pixel.

FIG. 28 illustrates the next step, where with both highlight areasidentified, the horizontal distance between their centres is used as ameasure of the range.

Stimulation

A lookup table is used to calculate the absolute pupil diameter inmillimetres from the measures of pupil pixel diameter and range. When avalid pupil measurement has been made, the system can start astimulation cycle to obtain the pupil constriction response curve afterstimulus by a bright white light source. The LED 14 generates whitelight In a stimulation cycle the LED 14 is energised. In this example,the period during which the LED 14 is energised for approximately 600ms.

During the stimulation cycle, the pupil diameter is continuouslymeasured and recorded using the above described algorithm whilst thewhite LED is energised. A graph, illustrated in FIG. 19, of the pupildiameter is then drawn, a typical response curve is shown below.

Where the following measurements can be taken;

L Latency (ms) Time between start of stimulus and beginning ofcontraction A Contraction Difference between the mean post-stimulusamplitude (mm) diameter and minimum per-stimulus diameter Tc ContractionTime from end of latency to minimum pupil time (ms) diameter

In the case of a hand-held pupilometer as described with reference toFIGS. 20 to 22, the graph may be displayed on the display 102 of thehand-held device, or on a VDU.

The response curve can be used in itself in diagnosis, or the responsecurve can form part of an expert system, which may generate a diagnosis.

The invention allows for the calculation of the distance between thesurface of the eyeball and the camera. No spacer of fixed dimension isrequired to establish a pre-determined distance between the camera andthe surface of the eyeball.

Furthermore, there is no requirement for a patient being examined tokeep its head still, and look in a fixed direction. The pupil of thepatient being examined need not be aligned with the centre of thecamera. The pupilometer of the invention functions as long as theinfra-red LED's produce highlights in the vicinity of the pupil As wellas permitting examination of semi-conscious or unconscious patients, thepupilometer can be used on patients who cannot necessarily followinstructions, for example children, impaired individuals, animals, etc.Rather than assuming that the pupil is a dark area in the centre of theimage, the pupil finds the dark pupil anywhere in the image.

The invention provides a simple and relatively low cost device for usein a variety of operational situations. Further, it provides a reliableand objective means of assessing pupil response.

1. A pupilometer comprising image capturing means, illumination meanscomprising two spaced apart light sources, stimulation means, and imageprocessing software, wherein said illumination means generates and emitslight of a first wavelength, and said stimulation means generates andemits light of a second wavelength, and wherein said illumination meansis arranged to one or both sides of said image capturing means and, inuse, shines light towards an eyeball, wherein the image processingsoftware receives data from the image capturing means, and by processingsaid data according to an algorithm establishes the distance between thesurface of the eyeball and the image capturing means, whereinestablishing the distance between the surface of the eyeball and theimage capturing means includes detecting the pupil and measuring thesize of the detected pupil.
 2. A pupilometer according to claim 1,wherein establishing the distance between the surface of the eyeball andthe image capturing means includes finding highlights on the surface ofthe eyeball generated by the illumination means and calculating thedistance between said highlights.
 3. A pupilometer according to claim 1,wherein the wavelength of the light generated by said illumination meansis in the infra-red spectrum.
 4. A pupilometer according to claim 3,wherein each light source is an infra-red light emitting diode.
 5. Apupilometer according to claim 1, wherein the image capturing means hasan optical axis, and wherein the two spaced apart light sources shinelight in a direction substantially parallel to the optical axis of theimage capturing means.
 6. A pupilometer according to claim 1, whereinsaid stimulation means comprises a light emitting diode generating andemitting light in the visible spectrum.
 7. A pupilometer according toclaim 1, wherein said image capturing means comprises a camera.
 8. Apupilometer according to claim 1, further comprising an optical filtermounted on the image capturing means.
 9. A pupilometer according toclaim 8, wherein the optical filter passes only light of the firstwavelength.
 10. A pupilometer according to claim 7, wherein said cameragenerates a video signal.
 11. A pupilometer according to claim 7,wherein said camera is a complementary metal oxide semiconductor device.12. A pupilometer according to claim 1, wherein said image detectionmeans further includes a micro-controller including a micro-processor.13. A pupilometer according to claim 1, further comprising an analogueto digital converter arranged between said image capturing means andsaid micro-controller.
 14. A pupilometer according to claim 1, furthercomprising memory means.
 15. A pupilometer according to claim 1, furthercomprising data input means and display means.
 16. A pupilometeraccording to claim 1, further comprising an interface for linking saidpupilometer to an external computer.
 17. A pupilometer according toclaim 1, wherein said pupilometer is a hand-held device, wherein saidhand-held device mounts said image capturing means, illumination means,stimulation means, image processing software, data input means, displaymeans, a computer interface, said hand-held device including a handgrip.
 18. A pupilometer according to claim 17, wherein, in use, the userviews the image of the eye displayed on the display means, the imagehaving been captured by said image capturing means and processed by saidimage processing software.
 19. A pupilometer according to claim 1,further comprising a power supply consisting of a battery.